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DETAILED ACTION 

Response to Amendment 

1 . In response to the office action from 04/14/2010, the Applicant has submitted an 
amendment filed on 07/14/2010, amending Claims 1, 35 and 45 and arguing to traverse the art 
rejections based on the limitations stated on Pages 8-1 1 of the Amendment. Applicant's 
arguments have been fully considered, but are moot in view of the new ground(s) of rejection. 

2. Upon further consideration, Claims 35 and 38 have been also rejected under 35 U.S.C. 
101 since the machine- readable-medium claims can be broadly interpreted as carrier waves. For 
more details see the 101 rejections below. 

Continued Examination Under 37 CFR 1.114 

3. A request for continued examination under 37 CFR 1.114, including the fee set forth in 
37 CFR 1.17(e), was filed in this application after final rejection. Since this application is 
eligible for continued examination under 37 CFR 1.1 14, and the fee set forth in 37 CFR 1.17(e) 
has been timely paid, the finality of the previous Office Action has been withdrawn pursuant to 
37 CFR 1.1 14. Applicant's submission filed on 07/14/2010 has been entered. 

Claim Rejections - 35 USC § 101 

4. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or 
any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and 
requirements of this title. 
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5. Claim 35 is directed to a computer readable medium storing processor executable 
instructions that is not limited to a tangible, and thus, statutory medium. The United States 
Patent and Trademark Office (USPTO) is obliged to give claims their broadest reasonable 
interpretation consistent with the specification during proceedings before the USPTO. See In re 
Zletz, 893 F.2d 319(Fed. Cir. 1989) (during patent examination the pending claims must be 
interpreted as broadly as their terms reasonably allow). The broadest reasonable interpretation of 
a claim drawn to a computer readable medium (also called machine readable medium and other 
such variations) typically covers forms of non-transitory tangible media and transitory 
propagating signals per se in view of the ordinary and customary meaning of computer readable 
media, particularly when the specification is silent. See MPEP 21 1 1 .01 . When the broadest 
reasonable interpretation of a claim covers a signal per se, the claim must be rejected under 35 
U.S.C. 101 as covering non-statutory subject matter. See In re Nuijten, 500 F.3d 1346, 1356-57 
(Fed. Cir. 2007) (transitory embodiments are not directed to statutory subject matter) and Interim 
Examination Instructions for Evaluating Subject Matter Eligibility Under 35 U.S.C.jlOl, Aug. 
24,2009; p. 2." In the present case, the computer readable medium is only generally/broadly 
recited in the specification/original claims (Original Claim 35). Thus the scope of "computer- 
readable medium" broadly includes signal-based mediums. A signal does not fall within one of 
the four statutory categories of invention (i.e., process, machine, manufacture, or composition of 
matter) because it is an ephemeral, transient signal and thus is non-statutory. Since the scope of 
"computer-readable medium" includes these non-statutory instances, claim 35 is directed to non- 
statutory subject matter. 
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Dependent Claim 38 fails also to overcome the 35 U.S.C. 101 rejections directed to 
Claim 35, and thus are also rejected for being drawn to non-statutory subject matter. 

Claim Rejections - 35 USC §103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

7. Claims 1, 4, 6, 8, 10, 14, 35, 38, 45-49 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Polcyn (U.S. Patent: 6,865,258) in view of Cox et al. (U.S. Patent: 6,192,339) 
and further in view of Sibal et al. (U.S. Patent Application 2003/0182622), hereinafter referred 
to as Polcyn, Cox and Sibal. 

With respect to Claims 1, 35, 45, Polycin discloses: 

A method, machine -readable medium having instructions which when executed cause a machine 
to, and system (Method, System and Computer Readable Medium, Polcyn, Col. 8, Line 62- 
Col. 9, Line 12, Col. 17, Lines 47-60, Col. 18, lines 1-13, and see also Cox, Col. 3, Lines 13- 

35) comprising: 

receiving at a server computer system a client request from a client computer device via a 
network (calling party transcription request and EMS 206 in communication via network 
204, Col. 7, Lines 56- Col. 8, Line 28); 



Application/Control Number: 10/534,661 Page 5 

Art Unit: 2626 

interpreting the client request including identifying a selection of at least 
one of a plurality of web interaction modes (EMS 206 may comprise voice capture capability, 
voice record capability, voice play capability, voice recognition, DTMF recognition, Col. 10, 
Lines 10-19), each of the plurality of web interaction modes to perform interpretation of content 
being transmitted between the server computer system and the client 

computer device (the communicating party routed via network 204 to EMS 206 where EMS 
206 comprises capability to receive image, fax, video, email; various forms of data may be 
communicated to the EMS 206 such as audio data, DTMF data, fax data, textual data, Col. 
10, Lines 6-19, 37-61); and 

identifying a web interaction mode selected by the client computer device (the transcription 
interface application monitors the transcriber's activity and automatically adjusts the 
presentation of data to be transcribed according to such activity data type, Col. 12, 
Linesl8-35)j and performing speech processing based on the selected web interaction mode (the 
transcription application utilizes voice recognition where the segment may be automatically 
transcribed and displayed in the appropriate field of data entry screen, Col. 14, Lines 14- 
32, 48-59), wherein performing speech processing includes determining an active display 
element that is to be focused (transcription application determines the transcriber's focus by 
determining the position of the cursor, Col. 17, Lines 21-39) and identifying the active display 
element with its associated identifier (the transcription application identifies the appropriate 
message segment corresponding to the transcriber's focus at block 410, and the 
transcription application may begin the presentation of the data of the appropriate 
message segment, Col. 17, Lines 21-39). 
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Polcyn, however, does not explicitly disclose, but Cox discloses wherein 
the active display element includes an element upon which a speech input received from a 
user is focused, the speech input is received via the client computer device (Depending on 
the voice input, corresponding machine commands derived from transition command mapping 
216 are issued and the appropriate speech applications become focused and begin executing. ..a 
user may say: "switch to device control program " through microphone 402... ", and also see 
how transition command mapping 216 may utilize different semantics for its generated 
statements, such as "focus on application XYZ" or "execute application XYZ". Also, transition 
command mapping 216 may display to a local user a list of available speech applications to 
choose from, Cox, Col. 5, Lines 59-67, Col. 6, Lines 40-64, Col. 5, Lines 4-18, client computer 
device in Figure 4 and Figure 4 as a whole, Figure 3, elements 314, 316, Figure 2, central 
information object 200 and distributed computer systems 100 with remote usage tractability and 
servicing as server), 

receiving an utterance from a user, via the client computer device, once the active display 
element is focused (receiving voice input where transition command mapping 216 may utilize 
different semantics for its generated statements, such as "focus on application XYZ" or "execute 
application XYZ", Cox Col. 5, Lines 4-18, Figure 4 as a whole and also Figure 3, elements 314, 
316 ), and, if the utterance matches the speech input, transmitting the identifier to the 
server computer system so that speech recognition is performed ( " ...support remote usage 
for both applications for the remote capability attribute... ", "...central information object 
200... " according to voice input matching of the focused application XYZ , Col. 6, lines 18-33, 
Col. 7, lines 21-48, Col. 5, Lines 4-18, client computer device in Figure 4 and Figure 4 as a 
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whole, Figure 3, elements 314, 316, Figure 2, central information object 200 and distributed 
computer systems 100 with remote usage tractability and servicing as server) \ 

performing speech recognition based on a relationship between the active display element 
and one or more speech elements {"...central information object 200 maintains information 
from listeners 202 and speech applications such as device control program 104 and answering 
machine program 106. ..Speech applications may either modify or retrieve the information stored 
in central information object 200 through signaling interface 206. ..Central information object 
200 may contain any of the following data, but not limited to, 1) currently focused speech 
application, 2) listening state of any speech recognition engine, 3) performance parameters and 
4) graphical user interface support information. Multiple speech applications utilize these data 
to comprehend the running states ofMASE 102 ...results in their seamless interactions with one 
another... ", "... device control program 1 04 and answering machine program 1 06 ... " ; 
" ... transition command mapping 216 at this point likely contains "switch to device control 
program " and "switch to answering machine program " items . Depending on the voice input, 
corresponding machine commands derived from transition command mapping 216 are issued 
and the appropriate speech applications become focused and begin executing.. .a user may say: 
"switch to device control program " through microphone 402... ", " ...multiple listeners 202 to be 
active simultaneously, but limit to a single instance of speech application per listeners 202. As a 
result, multiple distinct speech applications can coexist simultaneously ...all applications have 
access to the state information of other applications and the environment... ", Col. 3, lines 60-67- 
Col. 4, lines 1-8, lines 54-67, Col. 5, lines 36-52, Col. 7, line 53-Col. 8, line 6, Col. 5, Lines 59- 
67, Col. 6, Lines 40-64, Figure 4 as a whole and also Figure 3, elements 314, 316, Figure 2, 
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central information object 200 and distributed computer systems 100 with remote usage 
tractability and servicing ). 

Polycin and Cox are analogous art because they are from a similar field of endeavor in 
facilitating improved web accesses applications via speech recognition. Thus, it would have 
been obvious to a person of ordinary skill in the art, at the time of invention, to modify the 
teachings of Polycin with the technique for commanding visual and voice browsers in a common 
development platform and common environment taught by Cox in order to advantageously 
provide the user the desirability to dictation functionality in one product and device control 
functionality in another simultaneously in a seamless fashion, (Cox, Lines 51-55). 

Polycin in view of Cox, however does not explicitly disclose, but Sibal discloses wherein 
performing speech recognition includes retrieving a synchronization relationship between the 
one or more speech elements and the active display element to compose grammar of the one or 
more speech elements (synchronizing field/partial field inputs between voice and visual 
browsers so that the user can fill out different fields of a single form using a combination of 
both voice and visual/tactile mode; synchronizing the voice browser by pointing the voice 
browser to a dialog on the VXML page that corresponds to that field; "granularity"; multi- 
modal platform 110 communicatively connected to and from computer device 102 and 
Web server 120 storing and/or generating markup content, Paragraphs [0031], [0032], 
[0040], [0045], [0055]-[0058], [0135], Figs. 1, 5 and 7), and 

dynamically correcting the composed grammar of the one or more speech elements using a real- 
time speech recognition based on the synchronization relationship (field/partial field inputs 
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allowing the user to type "New" and speak "York"; typing the city "New York" and 
speaking the zip code "10001" according to "granularity", Paragraphs [0031], [0032], 
[0033], [0034], [0055]-[0058], [0135]). 

Polycin, Cox and Sibal are analogous art because they are from a similar field of 
endeavor in facilitating improved web accesses applications. Thus, it would have been obvious 
to a person of ordinary skill in the art, at the time of invention, to modify the teachings of 
Polycin in view of Cox with the technique for synchronizing visual and voice browsers to enable 
multi-modal browsing taught by Sibal in order to advantageously provide the user the usability to 
both browsers (visual and voice browsers) to interact with content simultaneously, (Sibal, 
Paragraphs [0004]-[0006]). 

With respect to Claims 4 and 38, Sibal discloses: 
wherein the focused active element comprises a hyperlink or a field in a form (the user can fill 
out a single field using a combination of voice and visual/tactile input (e.g., entering a city 
name by typing "New" followed by speaking "York"), Paragraphs [0031], [0032], [0025], 
[0027], [0055]-[0058], [0135]). 

With respect to Claims 6 and 46, Sibal discloses: 

further including: extracting speech features from a user speech input, wherein the user speech 
input is contained in the client request (synchronizing field/partial field inputs between voice 
and visual browsers so that the user can fill out different fields of a single form using a 
combination of both voice and visual/tactile mode, Paragraphs [0031], [0032], [0040], 
[0020], [0022], [0024], [0025], [0027], [0055]-[0058], [0135], Figs. 1, 5 and 7). 
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With respect to Claims 8 and 47, Sibal discloses: 

further including: receiving a session message at the server computer system to initialize a 
connection between the server computer system and the client computer device, wherein the 
session message includes an internet protocol (IP) address of the client computer device, a device 
type of the client computer device, a voice character of a user responsible for the user speech 
input, a language of the user input, and a default recognition accuracy requested by the client 
computer device (multi-modal platform 110 communicatively connected to computer device 
102 and Web server 120, client/server topology, web page 106 as portal page allowing the 
client to send request; computer device 102 requesting according to HTTP protocol; type 
of device; playing audio through speaker; multi-modal platform 110 configured to a "hit" 
of its own port as a signal to send information to visual browser, Paragraphs [0020]-[0022], 
[0024], [0025], [0027], [0028], [0031], 0032], [0038], [0055]-[0058], [0135]). 

With respect to Claims 10 and 48, Sibal discloses: 
further including: receiving a transmission message at the server computer system to exchange 
transmission parameters between the server computer system and the client computer device 
(multi-modal platform 110 communicatively connected to computer device 102 and Web 
server 120, client/server topology, web page 106 as portal page allowing the client to send 
request, Paragraphs [0020], [0022], [0024], [0025], [0027]). 

Also, Polycin disclose the communicating party routed via network 204 to EMS 206 
where EMS 206 comprises capability to receive image, fax, video, email; various forms of data 
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may be communicated to the EMS 206 such as audio data, DTMF data, fax data, textual data, 
(Col. 10, Lines 6-19, 37-61). 

With respect to Claims 14 and 49, Sibal discloses: 
further including: receiving an exit message at the server computer system to terminate a 
user session with the server computer system and the client computer device (logger module, 
time stamping, Paragraph [0278]). 

Conclusion 

8. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. See Form PTO-892. 

9. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Edgar Guerra-Erazo whose telephone number is (571) 270-3708. 
The examiner can normally be reached on M-F 7:30a.m.-5 :00p.m. EST. If attempts to reach the 
examiner by telephone are unsuccessful, the examiner's supervisor, James Wozniak can be 
reached on (571) 272-7632. The fax phone number for the organization where this application or 
proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private 
PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you 
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would like assistance from a USPTO Customer Service Representative or access to the 
automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Edgar Guerra-Erazo/ 
Examiner, Art Unit 2626 

/James S. Wozniak/ 

Supervisory Patent Examiner, Art Unit 2626 



